ATTORNEY DOCKET: 
090936.0525 



PATENT APPLICATION 



1 

SYSTEM AND METHOD FOR SIMULATING AUDIO COMMUNICATIONS 
USING A COMPUTER NETWORK 

TECHNICAL FIELD OF THE INVENTION 

This invention relates to the use of computer 
systems for simulating real world situations, and more 
particularly to the use of a computer system to simulate 
audio communications . 
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BACKGROUND OF THE INVENTION 

As the demand for simulation-based training and 
testing systems increases, so does the demand for more 
realism. A realistic simulation system should comprise 
5 at least three components: visual realism, tactile 
realism, and aural realism. Many tools are available 
that are directed to the first two components, but the 
aural component of realism is often given short shrift. 
As a result, many simulation systems provide a simulated 

10 environment with a great "look and feel", but lack a 
meaningful aural experience. 

As an example, a high level flight simulator might 
realistically simulate the cockpit of an aircraft and the 
feeling of flying. However, voices and other sounds 

15 attempted to be reproduced in the pilot's headset are not 
likely to sound like those actually transmitted over a 
radio channel. In such a system, the trainee pilot's ear 
is not being trained at the same level as his or her 
visual and tactile senses. 

20 
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BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the present 
embodiments and advantages thereof may be acquired by 
referring to the following description taken in 
5 conjunction with the accompanying drawings, in which like 
reference numbers indicate like features, and wherein: 

FIGURE 1 illustrates a simulation system in 
accordance with the invention. 

FIGURE 2 illustrates the programming architecture 
10 residing on the server of FIGURE 1. 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 illustrates a system 100 for simulating 
audio communications in accordance with the invention. 
The audio simulations provided by system 100 are 
5 significant to the simulation industry, but are also 

applicable in other fields, such as in the communications 
industry. With respect to the latter, by using discrete 
modules to simulate communications alternatives, the 
communications industry can make better decisions about 
10 which systems best meet design criteria. 

An example of aural realism provided by the 
invention is the replication of the sound of a UHF 
channel used in a flight simulation system. In this 
example, system 100 is used to accurately simulate voices 
15 that are received by a pilot over a UHF radio channel. 

As explained below, in one embodiment of the 
invention, the system is a client-server-client system. 
Any client can be a source and/or a destination. The 
server routes audio between the clients and introduces 

2 0 impairment effects as prescribed by simulation models and 

parameters. A source client is used to receive audio 
communications. The audio input is sampled and delivered 
to the server where it is filtered and injected with 
random noise and other effects, using digital signal 
25 processing (DSP) models. The impaired voice data is then 
delivered to a destination client. 

In real time applications, the audio distribution is 
real-time for maintaining the interactive fidelity of a 
training environment. In playback applications, the 

3 0 audio may be processed and recorded prior to playback. 
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Some applications might use a combination of real-time 
and playback audio. 

Real-time applications may use live participants, 
with some speaking into microphones and others listening 
5 through speakers to audio that is impaired in accordance 
with the invention. A playback application would permit 
"virtual participants' 7 to participate in the simulation 
and deliver impaired audio to one or more live recipients 
via headphones. 

10 System 100 may provide simulations of various audio 

transceivers, operating over wireless, metal wired, or 
fiber optic wired channels, or any combination of these. 
Each of these transmission mediums has unique audio 
characteristics, which are modeled by system 100. The 

15 simulated transceivers appear in many applications, 

mobile or stationary. Examples of transceivers includes 
those associated with aircraft, ground vehicles, or 
communications centers . 

System 100 is a distributed computer network system, 

20 having at least one server 101 and a number of client 
stations 102 and 103. Server 101 receives audio data 
from one or more transmitter client systems 102, applies 
DSP models to the data, and delivers the processed audio 
to one or more receiver client systems 103. Client 

25 systems may also be two-way (transceiver) systems, such 
as client systems 104 and 105 which send and receive 
audio. Although only four client systems are 
illustrated, system 100 may handle any number of client 
systems as is appropriate for its processing capacity. 

30 For purposes of this description, a "transmitter 

client" may be client system 102 or client system 104 or 
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105 when they are transmitting audio. Similarly, a 
"receiver client" may be client system 103 or client 
system 104 or 105 when they are receiving audio . 

In typical applications, such as flight simulation, 
the system 100 operates in real time with two-way 
clients. Live participants will transmit and receive 
audio data, via their respective client systems and the 
server 101. Any number of participants tuned to the same 
channel can send and receive audio. 

Client systems 102 - 105 have appropriate hardware 
and software for sampling input audio and playing 
received audio data. Client systems may be conventional 
personal computer type devices, including laptops and 
PDA's, equipped with conventional off-the-shelf sound 
cards and player devices, including microphones and 
speakers. Depending on the application, headphones (as 
illustrated in FIGURE 1) rather than speakers may be 
used. 

Server 101 is equipped with microprocessing devices, 
programmed to perform the tasks described herein. Server 

101 also performs standard computer server functions, 
such as receiving and satisfying requests for data from 
clients 102 and 103. 

FIGURE 2 illustrates the programming residing on 
server 101. As illustrated, the processing of audio data 
is performed by server 101, which relieves client systems 

102 and 103 of that processing burden. 

In the example of this description, server 101 is in 
data communication with clients 102 and 103 via an 
Ethernet connection. Thus, network 104 is a local area 
network. However, the same concepts could be applied to 
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a wide area network, such as one using a TCP/IP protocol. 
The communications links of network 104 may be wired or 
wireless. Each client 102 or 103 is equipped with a 
network interface card for data communications over 
network 104. A 100 Mbps Ethernet connection is an 
example of a suitable data speed. Various methodologies 
and protocols may be used for communicating the audio 
data across the network. An example of a suitable 
protocol is VoIP (voice over Internet Protocol) . 

Transmitting client 102 receives and stores sampled 
audio data. As an example, client 102 may be equipped 
with a microphone and soundcard (printed circuit board) 
for receiving and sampling (digitizing) spoken words. 

Client 102 then delivers the sampled audio data to 
server 101 via network 104. In the example of this 
description, data communications are via data packets. 
Server 101 processes the audio data by applying the 
simulation models of engine 21. 

Server 101 determines which receiver clients 103 are 
to receive the processed audio data, then delivers the 
audio data to those clients 103. Client 103 is equipped 
with a soundcard (printed circuit board) , which it uses 
to convert the processed audio data to analog data, which 
is played to the user of client 103, using headphones or 
speakers . 

Client systems 102 - 105 may be conventional 
computer workstations, having conventional processing and 
memory capability. As explained above, the special 
equipment for implementing the present invention includes 
audio equipment such as speakers, microphones, and sound 
cards . 
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Referring specifically to Figure 2, which 
illustrates the programming architecture of server 101, 
server simulation programming 20 resides on server 101. 
This programming 2 0 oversees the operation of the audio 
simulation engine 21 (including impairment models and 
transceiver models discussed below) , application 
interface 24, and the server- side graphical user 
interface (GUI) 25. Depending on the sophistication of 
the simulation programming, the audio simulation engine 
21 may be used in conjunction with a tactile simulation 
engine 22 and/or a visual simulation engine 23. 

The transceiver models and impairment models of 
audio simulation engine 21 are implemented with DSP 
(digital signal processing) programming. The DSP 
functions may be implemented with the main CPU of server 
101, or optionally, with special DSP devices operating in 
conjunction with the CPU of server 101. 

In general, the transceiver and impairment models of 
engine 21 are created by analyzing a real world 
communications system and its effects on transmitted 
signals. Their processing is used to model 
characteristics of real world radio communications 
channels. The processing is then used to impose those 
characteristics on a signal, so that the signal simulates 
a signal that was transmitted through those channels. 
Thus, the model is used to replicate the sound at the 
output end of a radio communications channel. 

Transceiver models model the type of radio or other 
transceiver being simulated. Typically, both the 
receiving and transmitting client systems are transceiver 
systems, but in certain embodiments, a transceiver model 
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could be of a receiver-only or transmitter-only 
equipment. A transceiver model may be as simple as a 
bandpass filter. The transceiver power is an example of 
one parameter to the model . 
5 Transceiver models include models of automatic gain 

control circuits and squelch circuits. Signal 
sensitivities (how low a signal can be received) are used 
for modeling these circuits. Bandpass filters are used 
to model the frequency response of a transceiver. 

10 Impairment models model the effects of the 

transmission medium, such as by modeling channel fading, 
multi-path fading, propagation delay, noise (including 
Gaussian), Doppler shift effects, and gain control. For 
wireless communication simulations, in addition to 

15 filtering and noise injection, the impairment models also 
simulate the effects of line-of -sight , propagation 
losses, signal fading, and other effects characteristic 
of wireless communications. The impaired audio could 
include any desired background noise, in addition to the 

2 0 communicated voice audio, such as engine noise, wind 
noise, artillary noise, etc. 

A feature of the invention is that the impairment 
models may operate dynamically, in response to simulated 
conditions. For example, for simulating aircraft audio 

2 5 communications, an aircraft may be flying at a certain 

speed and be a certain distance away from a control tower 
with which it is communicating. The speed and distance 
may be used as parameters to the model, which will 
process the audio data differently depending on these 

30 parameters. For purposes of this description, parameters 
such as speed and distance of a client system (i.e., a 
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simulated aircraft) are referred to as "dynamic simulated 
entity data". 

Multipath impairment models use as input, the 
velocity of the entity (aircraft, vehicle, etc.) that is 
5 receiving the transmission and the radio frequency. 

Doppler effect impairment models use the velocity and the 
angle between the transmitter and receiver. Channel 
noise models use an additive Gaussian white noise 
generator whose level is controlled by the path gain. 
10 Path gain is represented as a fraction of signal power 
based on the distance between the transmitter and 
receiver, the power of the transmitter, and the carrier 
frequency. 

As an alternative to sampling real world audio 

15 input, the models of server 101 could be used to generate 
audio data. Various techniques for artificially 
generating voice and other audio sounds could be used. 

The application program interface (API) 24 controls 
the interaction of the audio simulation engine 21 and the 

20 visual and tactile simulation engines 22 and 23. The API 
24 operates in conjunction with the tactile simulation 
engine 22 so that tactile entries made at a client system 
102 - 105 may be used to control the audio simulations. 
For example, an operator at a client system may select a 

25 particular simulation environment, which includes a 
particular type of transceiver. Additional tactile 
selections by the operator could include channel and 
volume settings or any other type of control found on a 
radio or other audio equipment. The API 24 may also be 

3 0 used to provide visual displays to correspond with the 
tactile selections and the audio environment. For 
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example, a visual display of a radio console could be 
used to depict controls (such as switches, knobs, and 
indicator lights) for volume, noise level, signal 
strength, or other audio qualities, or to depict current 
5 levels of these qualities. 

In the case where programming 2 0 includes visual and 
tactile simulation engines 22 and 23, client systems 103 
- 105 would then have appropriate graphical user 
interfaces for providing these visual displays and 

10 receiving tactile inputs. However, a feature of the 

invention is that the audio processing of engine 21 is 
performed at the server, which permits system 100 to be 
implemented with a minimum of processing at client 
systems 102 - 105. Specifically, for purposes of 

15 implementing the audio simulation provided by engine 21, 
client systems 102 - 105 need only a sound card, and 
speaker and/or microphone and whatever software is 
required to receive and play audio data. 

The interrelated audio, visual , and tactile 

2 0 simulations described in the preceding paragraph relate 

to simulated communications and related hardware. System 
100 and programming 2 0 may be further enhanced to include 
(or work in conjunction with) additional simulation not 
related to audio. For example, in the case of a flight 
25 simulator, these additional controls might include flight 
controls and cockpit views . 

Graphical user interface (GUI) 25 is used to 
configure and control the simulation engines 21-23. A 
system administrator will specify the models used for a 

3 0 particular simulation, as well as various impairment 

parameters, and other operational characteristics. 

AUS01 : 331342 . 1 



ATTORNEY DOCKET 
090936 . 0525 



12 



PATENT APPLICATION 



System 10 and its programming can be fully 
implemented using a combination of conventional computing 
devices and conventional programming languages and tools. 
The programming described herein can be implemented with 
5 C++ language to enhance its ability to interface with 
existing non-audio simulation systems. 



AUS01 :331342 . 1 



